1. Field of the Invention
The present invention relates to a signal processing apparatus, a signal processing method, a computer program, an image processing apparatus, and an image capturing apparatus processing a signal of an image captured by an image sensor such as a CCD or a CMOS, and more particular, to a signal processing apparatus, a signal processing method, a computer program, an image processing apparatus, and an image capturing apparatus processing the signal of a RAW image which is not processed particularly in a de-mosaic process or the like.
2. Description of the Related Art
In recent years, digital cameras performing digital coding on images captured by an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) have come into wide use, instead of silver salt cameras photographing an image using a film or a photographic plate. The digital cameras have the advantage of having no life problem of a film since the digital cameras can store images subjected to digital coding in a memory and can process or manage the images with a computer.
Both the CCD image sensor and the CMOS image sensor also have a configuration in which pixels (photodiodes) arranged in a 2-dimensional form convert light into charges using the photoelectric effect. In numerous digital cameras, since a single plate type color is used, only single color information is used among color information on red (R), green (G), and blue (B) in each pixel. Therefore, a “de-mosaic” process has to be performed to obtain a full color image by collecting insufficient color information from peripheral pixels for each pixel when an image is photographed and by interpolating the color information. Then, the completed image is compressed in conformity with a general image format such as JPEG (Joint Photographic Expert Group) or TIFF (Tag Image File Format) and is stored.
However, the precision of the de-mosaic process has a great influence on the image quality of the completed image. Moreover, since a white balance (color temperature) or the like of the image subjected to the de-mosaic process is fixed, correction may not be performed easily. Since an image recording format such as JPEG is supposed to be processed after image-processed data is input, the image recording format may not be used for recording the signal of a RAW image. Therefore, a function of storing, as a file, the signal (that is, the output signal of an image sensor) of the RAW image which is not subjected to the de-mosaic process is necessary mainly in high-performance cameras. Since the signal of the RAW image is generally subjected to non-compression or reversible compression, the signal of the RAW image has a very large file size, compared to JPEG. However, when the RAW image can be suppressed so as to have the same file size as that of JPEG with high quality (see FIG. 15), the function of a camera is improved and thus competitiveness is improved in the market. Comparison between a RAW image and a JPEG image is shown in the following table.
TABLE 1ADVANTAGESDISADVANTAGESRAWNON-APPLICATION OF IMAGERELATIVELY LARGEPROCESSING, LARGE DEGREESIZEOF FREEDOMJPEGSMALL SIZE, IMMEDIATELERECOVERY ISDISPLAYDIFFICULT AFTERIMAGE PROCESSING
In an image compression method such as JPEG, the amplitude of an image signal is converted into a frequency space using DCT and an amount of information is reduced based on the characteristics of the sense of vision. On the other hand, a method of compressing the signal of the RAW image on the focus of a noise of data was suggested in the past.
For example, there was suggested a signal conversion apparatus cutting off and quantizing an alternating current component with an amplitude equal to or smaller than an average amplitude (hereinafter, referred to as a “noise level”) of a noise (for example, see Japanese Patent No. 2548005). An output signal x from an image sensor is the number of electrons (that is, discrete value) corresponding to lightness and the noise level is varied depending on the number of electrons. Specifically, since the noise increases with an increase in the lightness, the noise level also increases. In the signal conversion apparatus, an output signal y determined by Expression 1 can be obtained for the image signal x.
                    y        =                              f            ⁡                          (              x              )                                =                      c            ⁢                                          ∫                b                x                            ⁢                                                [                                      1                    /                                          n                      ⁡                                              (                        ξ                        )                                                                              ]                                ⁢                                  ⅆ                  ξ                                                                                        (        1        )            
In Expression 1, ξ is an integration variable corresponding to the value of the input signal x, n(ξ) is a standard deviation (average amplitude) of a noise superimposed in an input signal which is expressed as a function of the integration variable ξ, b is a signal voltage (constant) when the lightness is 0, and c is a predetermined constant value. In Expression 1, a noise level ny of y is a constant independent of y, as expressed in Expression 4, when the noise occurring in the signal conversion apparatus can be ignored for the entire noise contained in y. Therefore, when the signal y is quantized in the next stage, LSB (Least Significant Bit) is quantized to correspond to c of Expression 4 and thus a digital signal y is obtained. At this time, the capacity necessary in a transmission path of the output signal y is smaller than the capacity necessary in a transmission path of the input signal x.ny=(dy/dx)n=c  (2)
To increase a compression ratio of the signal of the RAW image, the constant c may be decreased in Expression 1. That is, the amount of data can be reduced by decreasing the constant c. However, the decrease in the amount of data sacrifices image quality. Moreover, when the constant c is decreased, unnatural gray scales may be noticeable in the image subjected to signal converting. Japanese Patent No. 2548005 does not disclose this phenomenon. Moreover, since occurrence of the unnatural gray scales may not be predicted, the unnatural gray scales have to be adjusted based on experience.
In FIGS. 16A to 16D, the result obtained by performing signal converting on an image sample by Expression 1 as varying the constant c is shown. It can be observed that the unnatural gray scales appear with the decrease in the constant c. FIG. 17 shows a luminance distribution in which the same image sample gradually varies from lightness to darkness when the same image sample is subjected to integralization in the luminance direction and is further subjected to smoothing. Here, the horizontal axis represents position and the vertical axis represents luminance. As shown in FIG. 17, it is understood that the very small constant c in Expression 1 may cause the unnatural gray scales (solarization).